TouchIt: Real-Time Algorithm for Detecting Physical Contact in Video

We present an algorithm for fast depth segmentation and its application to detection of physical contact with a geometrically static surface in video. The algorithm is insensitive to lighting variations in the scene and presents a robust method for real-time depth segmentation. In this paper we discuss a technique for computing a disparity map for a surface given analytically. It is based on a two-camera setup, which utilises a camera-to-camera geometry that can be easily obtained by a number of off-the-shelf computer vision libraries. The algorithm uses pre-computed disparity maps to detect regions of the scene that are not at predetermined depths. The form of the algorithm allows it to take advantage of the single instruction, multiple data (SIMD) instruction set extensions that have recently become commonly available in consumer-grade microprocessors. The algorithm discussed in the paper is applied to the problem of detecting contact between a foreground object (a hand) and a geometrically static background (a display surface) in real-time video sequences. Due to the geometric nature of the segmentation algorithm, the touch detector is invariant to lighting, color, and motion. It therefore is applicable to interactive frontand back-projected displays. General Terms Stereo segmentation